Local dimension and finite time prediction in spatiotemporal chaotic systems 
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We show how a recently introduced statistics [Patil et al, Phys. Rev. Lett. 81 5878 (2001)] 
provides a direct relationship between dimension and predictability in spatiotemporal chaotic sys- 
tems. Regions of low dimension are identified as having high predictability and vice-versa. This 
conclusion is reached by using methods from dynamical systems theory and Bayesian modelling. We 
emphasize in this work the consequences for short time forecasting and examine the relevance for 
factor analysis. Although we concentrate on coupled map lattices and coupled nonlinear oscillators 
for convenience, any other spatially distributed system could be used instead, such as turbulent fluid 
flows. 
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I. INTRODUCTION 



Systems formed by aggregates of parts that interact 
in a nonlinear way are prototypes of complex behaviour 
in physics, biology and economics. An often successful 
approach in the analysis of such systems is to use tools 
from dynamical systems theory applied locally at points 
of spatially distributed configurations. In many cases lo- 
cal nonlinearities lead to unpredictable chaotic evolution 
where short time forecasts are still feasible. An instance 
of this situation is atmospheric research where, for obvi- 
ous reasons, prediction is often the most important goal 
to be achieved. In general, for large spatially distributed 
systems, it is highly desirable to have a simple diagnos- 
tic tool to identify regions of predictable behaviour. The 
main result of this paper is to show that certain spa- 
tial regions do indeed yield better forecasts than other 
locations. To elaborate on this theme we use the con- 
cept of BV dimension, introduced in El in the context 
of the Earth's atmosphere, as the tool to identify re- 
gions of high dimensionality. Our objective is then to 
show that this dimensional estimate is directly related 
to our ability to make short time forecasts. A recently 
introduced Bayesian approach, the cluster weighted mod- 
elling, CWM §, is one of the methods on which we base 
our conclusions. We start from a simple prediction al- 
gorithm H where the main idea of the paper is readily 
recognized. The more sophisticated CWM approach in 
conjunction with the simpler prediction algorithm will 
provide a clear link between dimension and our ability to 
forecast the behaviour of deterministic systems. In par- 
ticular, Bayesian approaches also permit the calculation 
of the variance (confidence intervals) of forecasts. In Ref. 
Q we find a similar approach where a measure of pre- 
dictability is based on the variance under the evolution 
of suitably defined functions in embedding space. Such 
approach requires long evolution times which cannot be 
afforded in the present context and, as will be seen herein, 
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this problem is circumvented by the use of the CWM. 

The concept of correlation dimension |], |(| has some 
bearing on this subject since high dimensional systems 
cannot be easily distinguished from systems with domi- 
nant stochastic component, where predictability is known 
to be low. Here by prediction we mean the existence of 
a deterministic map which can be successfully used to 
make short time forecasts, in the least square sense or 
using some more general likelihood function. 

The paper is organized as follows. In Sec. H, we 
present the basic aspects and definitions about the BV 
dimension, factor analysis and the necessar y to ols to re- 
late dimension and predictability. In Sec. Ill, the spa- 
tiotemporal systems given by coupled logistic maps and 
coupled Lorenz systems are discussed and the results of 
the simulations are presented. Our concluding remarks 
are outlined in Sec. |y|. 



II. LOCAL DIMENSION AND 
PREDICTABILITY 



The notion of dimension used herein is based on the 
concept of bred vectors. They are constructed in a sim- 
ilar way as Lyapunov vectors but in practical applica- 
tions they differ in two aspects. Firstly, for bred vectors 
there is no global orthonormalization and secondly, they 
are finite amplitude, finite time vectors. Such properties 
facilitate the calculation of bred vectors and yield an ef- 
ficient identification of regions were short time forecasts 
are feasible. 

Consider a 2D spatially distributed system whose state 
at a given time t\ is defined over a collection of points 
(i, j). Here we take the M—l nearest neighbours for each 
point (ijj) in a square lattice with M = 25, as illustrated 
in Fig. |l|. Logistic maps are one variable dynamical sys- 
tems and in order to specify the corresponding state at 
a point including its neighbours we need an M dimen- 
sional state vector. Considering the x — y coordinates 
of the Lorenz system specified at a suitable constant z 
hyperplane, the state vector in this case requires 2M 
components. In general the state vector, either M di- 
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FIG. 1: Schematic diagram showing the choice of nearest 
neighbours at site for local dimension. The bred vec- 

tors are the dynamical variables associated with these sites. 



mensional or 2M dimensional, will be called bred vector. 
Now generate k distinct perturbations of the state start- 
ing at to < t\ obtaining k local bred vectors. The k x k 
covariance matrix of the system is just C = B T B, where 
B is the M x k (or 2M x k for the Lorenz system) ma- 
trix of local bred vectors each normalized to unity. In 
this paper we will fix k = 5. 

We order the eigenvalues of the covariance matrix as 
Ai > A2 > . . . > Afc and define the singular values of B 
as (Tj = \/A7. Here the connection with factor analysis is 
clear since the eigenvalues of the covariance matrix 
give an idea of the local linear independence of the k lo- 
cal bred vectors. An effective dimension of the space of 
bred vectors can be obtained by fixing a threshold value 
corresponding to the highest I eigenvalues, as is done in 
principal components analysis. Thus an approximation 
of the data, supposing zero average for simplicity, is con- 
tained in the product TL were T = BL T is called the 
factor and L the loading matrix of dimension I x k. The 
rows of the loading matrix are the components of the 
eigenvectors corresponding to the dominant eigenvalues. 
Clearly there is an arbitrariness in the stipulation of I 
and this ambiguity is absent when using the concept of 
BV dimension. 

The eigenvalues Ai represent the amount of variance in 
the set of the k unit bred vectors. In order to estimate 
unambiguously the value of the threshold one defines the 
following statistic [fij 
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As each of the k bred vectors is normalized to unity, tp 
assumes values in the interval (0, k). Examples of the 
values of this statistic for several distributions of bred 



vectors can be found in JJ|. A property of the statistic 
just defined is its robustness under noise or numerical 
errors. It can be used to determine the dominant eigen- 
values I: just take it to be the smallest integer bigger than 
■ip. In this sense an approximation to the bred vectors is 
obtained as the product of the corresponding factor and 
the loading matrix. 

We describe now the criteria and tools that will be 
implemented in the next section to spatially distributed 
systems in order to obtain a connection between dimen- 
sion and predictability. We start with a simple prediction 
algorithm which is very useful in the analysis of nonlin- 
ear time series ||. To describe this criterion, consider 
a scalar time series obtained from a component of the 
dynamical variables, say, the ir-component, defined at a 
given site in the 2D lattice. Using delay coordinates one 
considers neighbours to within a distance smaller than 
S to a given point in embedding space. For the present 
analysis the choice of embedding dimension 3 is adequate. 
By propagating the corresponding neighbours of a given 
vector in embedding space and averaging them we obtain 
a forecast of the vector. The more sophisticated CWM 
Q is a Bayesian approach where the embedded time se- 
ries is used to build local Gaussian models. One of the 
advantages of this method is that it goes beyond point 
prediction since it includes error bars (confidence inter- 
vals) when estimating the future average value. The idea 
is to expand the joint probability distribution p(y, x) in 
terms of local Gaussian models. Using the x components 
at each site for definiteness, the value to be predicted y 
is taken as x(n + It) (e.g. with embedding dimension 
to = 3 and delay r), given the vector of delayed compo- 
nents x = {x(n), x(n + t),?i} . The joint distribution is 
used to compute the average predicted values, (y|x) and 
their variances, ((y — (y|x)) 2 |x). This provides a criterion 
for predictability since higher variance is associated to 
lower predictability. The use of uncertainty as measured 
by the variance is employed in Q to obtain a definition of 
predictability associated to chaotic systems or stochastic 
processes and such method is used to distinguish between 
these two modes of evolution. Such distinction has also 
been made in the context of time series analysis using 
the CWM approach §. 



III. LOCAL DIMENSION IN 
SPATIOTEMPORAL CHAOTIC SYSTEMS 

A. Coupled logistic maps 

We begin with the case of two dimensional coupled 
map lattices consisting of logistic maps || . 

(2) 



with 



f(x)=fix(l-x), £€(0,1), MG(0,4), (3) 



3 



where i,j = 1,2,...N and e represents the coupling 
strength. 

In this simulation we use N = 50, fx = 4, e = 0.4 
and periodic boundary conditions. The reference spatial 
variables x l ' J , i,j = 1,2 ... N, are obtained by evolving 
system (^) from random initial conditions. The num- 
ber of iterations is chosen to be 5015 so that transients 
are removed. We generate additional spatial variables by 
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FIG. 2: Gray scale plot showing the regions of low (dark) and 
h^gh (bright) local dimensions for the coupled logistic maps 
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FIG. 3: Simple finite prediction error for the coupled logistic 
maps. The error remains minimum value for a; 2 ' 37 (low BVD 
point) and high values of error obtained for x 26 ' 7 (high BVD 
point) . 

adding small perturbations to the reference variables at 
time 5000 to measure the local instability of the coupled 
maps. Thus at time 5015 we have spatial distributions 
corresponding to the reference variable and four distinct 
perturbations. The local dimension at the spatial points 
are computed at this time value using the statistic defined 
in Eq. ([!]) as discussed in the previous section. Fig. ^ il- 
lustrates the results of the logistic maps (|J) calculations 
where dark regions correspond to low dimensions and 



bright regions represent high dimensions. We found that 
the local dimension has a minimum value, tp = 1.0323, 
at i = 2, j = 37 and a maximum value, ip — 2.3636, at 
i = 28, j — 7. We note that the maximum and minimum 
values at the spatial points (L j) are practically constant 
under evolution of system (|2|) a few steps forwards or 
backwards. 

In order to establish a connection between dimen- 
sion and predictability we analyse the time series at 
the points (2,37) and (28,7) discussed above. We ap- 
ply the simple prediction algorithm during the interval 
from n = 5015 to n — 5040. Fig. || shows the predic- 
tion errors, ri =1 x l ' J ■ — x 1 ' 1 , I, for x 2,37 and x 28,7 with 
' ' 1 orig pred " 

5 = 0.05. It is evident that the prediction error is small at 
the lattice point (2, 37) where the local dimension is low, 
and large at the point (28, 7) where the local dimension 
is maximum. In addition, similar results were obtained 
for points with other local dimension values. We found 
that the prediction error is consistently low in regions of 
low local dimension and high in the regions of relatively 
high local dimension. 

The Bayesian approach known as the cluster weighted 
modelling will provide more insights into the predictabil- 
ity issue. Particularly important is the relationship be- 
tween variance of predicted average value and the BV 
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FIG. 4: Prediction of time series for the coupled logistic maps 
by using cluster weighted modelling. The original (circles) 
and predicted values (filled circles with errorbars) of the time 
series at (a) low BVD location (2, 37) and (b) high BVD lo- 
cation (28, 7). 

dimension. We consider the average prediction (y|x) us- 
ing the conditional distribution obtained from the joint 
distribution p(y,x). As discussed in the previous sec- 
tion, we take y to be the x variable at time n + 2r and 
x = {x n ,x n+T ,n}; here we take r = 1. Fig. || presents a 
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series of predicted values, using always the two most re- 
cent original values, and the corresponding variance. In 
most cases analysed we found the following behaviour. 
Lattice sites with high BV dimension result in predicted 
values with larger variances than those with low BV di- 
mension, or either the prediction tend to fall outside the 
confidence interval defined by the variance of the future 
value. These facts are shown in Figs. ||(a) and (b) for 
the low and high BV dimensions, respectively 1.0323 and 
2.3636. In this case the simple prediction results are more 
compelling than the CWM since the difference between 
highest and lowest BV dimension is not big enough. In 
the next subsection we discuss another system where the 
maximum BV dimension is about four times larger than 
the minimum dimension over the spatial distribution. In 
this case the conclusion that the uncertainty in prediction 
is related to dimension is even more forceful. 



B. Coupled Lorenz systems 

We next consider a two dimensional array of diffusively 
coupled Lorenz oscillators represented by the following 
equations |To| : 

Xi,j = v{Vi,j - SEi,j) + c{Ui-i,j + Vi+ij + Vi,j-i 
+Vi,j+i - 

— %i,jyi,j (4) 

where i,j = 1, 2, N, a = 16, R = 40, /3 = 4 and 
c = 1. We use 50 x 50 (TV = 50) oscillators with periodic 
boundary conditions. Equations (Q) are solved numeri- 
cally using fourth order Runge-Kutta method with ran- 
dom initial conditions. We actually consider the x and y 
components representing a map of the above system by 
the Poincare section taken at z = R— 1. The spatial dis- 
tribution of the local dimensions is calculated in a similar 
fashion as in the case of coupled map lattices discussed 
earlier. A low value of ip = 1.1447 is found at lattice 
point = (3,50), while a high value ip = 4.1226 is 

computed at (i, j) — (11,46). Fig. || shows the regions of 
different local dimension for the coupled Lorenz systems 
(0). The finite time prediction error in Fig. ^juses S ~ 0.8 
in the simple prediction algorithm. Here the error is cal- 
culated as 77 = y (x mig - z pred ) + (y Qrig - j/ pred ) . 

The results concerning the cluster weighted modelling 
are shown in Fig. 0. Here high local dimension imply a 
consistent increase in the variances represented by larger 
error bars. In this case there is a clear relationship be- 
tween predictability and the local BV dimension. 

IV. SUMMARY AND CONCLUSIONS 

Predictability is related to the uncertainty in the out- 
come of future events during the evolution of the state of 
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FIG. 5: Gray scale plot showing the regions of low (dark) and 
high (bright) local dimensions for the coupled Lorenz systems 
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FIG. 6: Simple finite time prediction error for the coupled 
Lorenz systems. The absolute error remains minimum at 
(3, 50) for which the BV dimension is low and high values 
of error obtained at (11, 46), the high BV dimension point. 



a system. We have interpreted the CWM as a tool to de- 
tect such uncertainty and used it in spatially distributed 
systems. The simple prediction algorithm in conjunction 
the CWM form a powerful set of methods to relate pre- 
dictability and dimension. Another tool based on the 
variance of future states of a system is also employed in 
Q where a level of predictability is defined and applied 
to distinguish between deterministic and stochastic be- 
haviour. Such distinction requires a propagation time 
longer than the short time behaviour used herein and 
the cluster weighted modelling is more appropriate in the 
present context. Both methods, however, CWM or [Q, 
are more than diagnostic tools and can be used to make 
real time predictions. 

The short time evolution used here is not only a re- 
quirement for predicting chaotic systems but it also guar- 
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FIG. 7: Prediction of the time series using cluster weighted 
modelling at (a) (3,50) and (b) (11,46). 



antees the consistency of our conclusions. This comes 
about since the primary step is to identify spatial points 
of small BV dimension and then to make short time fore- 
casts for the variables at these points. If the dimension 
changed substantially during the short time evolution 



then the relationship between dimension and prediction 
could not be maintained. In this work predictions are 
made over intervals of 20 or 25 units of time (cycles) and 
under such circumstances the value of BV dimension is 
practically constant. 

For deterministic evolution some systems are more pre- 
dictable than others and this can be measured by Lya- 
punov exponents. However, these exponents are well de- 
fined only asymptotically and are not unique for finite 
time calculations fyl ff^, O, E3. In such cases bred 
vectors are the proper tool to use and in this paper we 
provided the connection between predictability and the 
value of the BV dimension. 

The concept of bred vectors is intimately related to 
the analysis of geophysical flows Jl], [l5). Other forecast- 
ing techniques could be envisaged for real applications 
other than the CWM fT|, 0. The main thesis of this 
work however will remain unaltered in such situations 
since the system is deterministic and the simple predic- 
tion algorithm or the CWM are essentially finite dimen- 
sional maps. Other algorithms would at most imply some 
differences for instance in the error functions with no im- 
pact in the main conclusions. In a further investigation 
we intend to apply the concept of BV dimension and the 
tools used herein to several configurations of fluids in the 
turbulent regime. 
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